Column ordering for input/output optimization in tabular data

ABSTRACT

Systems, methods, and computer-readable media for determining column ordering of a data storage table for search optimization are described herein. In some examples, a computing system is configured to receive input containing statistics of a plurality of queries. The computing system can then determine a new column order (i.e., layout) based at least in part on the statistics. In some example techniques described herein, the computing system can determine the new column order based at least in part on the hardware components storing the data storage table, storage system parameters, and/or user preference information. Example techniques described herein can apply the new column order to data subsequently added to the data storage table. Example techniques described herein can apply the new column order to existing data in the data storage table.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/579,522 filed on Dec. 4, 2017, which is a U.S. National StageApplication of PCT/US2016/035607, filed Jun. 3, 2016 which claimsbenefit of Chinese Patent Application No. 201510371595.6, filed Jun. 29,2015, and International Patent Application No. PCT/CN2015/080766, filedJun. 4, 2015, and which is incorporated herein by reference in itsentities. To the extent appropriate, a claim of priority is made to eachof the above disclosed applications.

BACKGROUND

As the world becomes more connected through technology, a greater numberof people need to store and access data. Such data is typically storedin two-dimensional tables comprising rows and columns. Traditionally,the data was organized in a row store format. As the tables haveincreased in size, however, more storage systems have moved to a columnstore format. Data storage tables now comprise thousands of columns ofdata, and continue to grow every day. As data is stored, new columns maybe added to the table. Because most systems store data sequentially asthe data is input, two unrelated columns of data may be stored adjacentto one another. In a table with thousands of columns, with relevantcolumns spaced far apart, the query can be very computationallyexpensive.

SUMMARY

This disclosure describes systems, methods, and computer-readable mediafor determining column ordering of a table for input/outputoptimization, such as, for search examples, a computing system isconfigured to receive input containing statistics of a plurality ofqueries. The computing system may then determine a new column order(e.g., table layout) based at least in part on the statistics. In someexample techniques described herein, the computing system can determinethe new column order based at least in part on the hardware componentsstoring the table, storage system parameters, and/or user preferenceinformation. Example techniques described herein can apply the newcolumn order to data subsequently added to the table. Example techniquesdescribed herein can apply the new column order to existing data in thetable.

It should be appreciated that the above-described subject matter mayalso be implemented as a computer-controlled apparatus, a computerprocess, a computing system, or as an article of manufacture such as acomputer-readable medium. These and various other features will beapparent from a reading of the following Detailed Description and areview of the associated drawings.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key or essentialfeatures of the claimed subject matter, nor is it intended to be used asan aid in determining the scope of the claimed subject matter. The term“techniques,” for instance, may refer to systems, methods,computer-readable instructions, modules, algorithms, hardware logicand/or operations as permitted by the context described above andthroughout the document. Furthermore, the claimed subject matter is notlimited to implementations that solve any or all disadvantages noted inany part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is set forth with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical items.

FIG. 1 is a block diagram depicting an example environment in whichexamples of column ordering for input/output optimization can operate.

FIG. 2 is a block diagram depicting an example computing device ofconfigured to participate in a column ordering for input/outputoptimization system.

FIG. 3 is a dataflow diagram depicting example module interactionsduring operation of column order evaluation and reordering forinput/output optimization.

FIG. 4 is a flow diagram that illustrates a process flow of determininga new column order using a column ordering for input/output optimizationsystem.

FIG. 5 is a flow diagram that illustrates a process flow of determininga seek cost in a column ordering for input/output optimization process.

FIG. 6 is a flow diagram that illustrates a process flow of determininga new column order using a column ordering for input/output optimizationsystem.

DETAILED DESCRIPTION Overview

The technologies described herein provide techniques and constructs toimprove the speed of input/output operations from tabular storage fordata analytics and processing, such as for providing a response to asearch query, by reordering the column order of a table. In someexamples, a computing system can receive statistics related to multiplequeries for information. The statistics can include a frequency of thequery (e.g., query A appeared 6 times over the last week), a list of thecolumns accessed per query (e.g., query A accessed columns 5, 13, and68), a time stamp for the query (e.g., query A appeared at 1615 GMT onJul. 17, 2014), a seek cost for the query (e.g., computationalcomplexity of the query), and/or other information related to the query.

In various examples, the computing system can assign a weight to eachquery based on the statistics. For instance, a query with a highfrequency can be weighted higher than another query with a lowerfrequency. Additionally or alternatively, the computing system canassign a weight based on user preferences. For instance, the computingsystem can receive an indication from a user that a particular query isof high importance to the user. The computing device can save the userpreference information, such as, for example, in a user profile, and canassign a higher weight to the query.

In some examples, the computing system can analyze the hardwarecomponents storing the table. The analysis can comprise determining aseek cost function for the hardware component. For instance, a tablestored on a hard disk drive (HDD) may be affected by a delay of trackseek (i.e., the delay of moving the head to a track or cylinder) and adelay of disk rotation (i.e., the time it takes for the desired sectorto rotate to the head). Two types of HDDs, HDD1 and HDD2, may havedifferent seek cost functions based on, for example, a disk rotationspeed. In various examples, the computing system can apply a seekdistance (e.g., a distance between columns) to the seek cost function ofthe hardware component to determine a seek cost for a query. Otherhardware components can have different cost functions that relate thecost of accessing different columns in the table.

Additionally or alternatively, the computing system can analyze one ormore system parameters. The one or more system parameters can comprise acolumn compression, a row size, a data type, a row type, and/or anyother parameters related to the data storage.

Based on the input statistics, user preferences, analysis of thehardware component and/or system parameters, the computing device candetermine a new order for the plurality of columns. This new order isoptimized for the query input statistics such that the totalinput/output cost to serve the query workload is minimized when storingthe data in the new order. In various examples, the computing system canapply the new order to data subsequently added to the table. In someexamples, the computing system can apply the new order to previouslysaved data by resaving the plurality of columns in the new order. Byreordering the columns of a table, whether including preexisting data orapplied to subsequently saved data, the time required for theInput/Output (I/O) process can be minimized, thereby having the effectof increasing the efficiency of the queries and decreasing the timerequired to respond to the queries, such as data analytics queriesand/or search queries.

Illustrative Environment

The environment described below constitutes but one example and is notintended to limit the claims to any one particular operatingenvironment. Other environments can be used without departing from thespirit and scope of the claimed subject matter.

FIG. 1 shows an example environment 100 in which ordering columns of atable for input/output optimization can be implemented. In someexamples, the various devices and/or components of environment 100include distributed computing resources 102 that can communicate withone another and with external devices via one or more networks 104.

For example, network(s) 104 can include public networks such as theInternet, private networks such as an institutional and/or personalintranet, or some combination of private and public networks. Network(s)104 can also include any type of wired and/or wireless network,including but not limited to local area networks (LANs), wide areanetworks (WANs), satellite networks, cable networks, Wi-Fi networks,WiMax networks, mobile communications networks (e.g., 3G, 4G, and soforth) or any combination thereof. Network(s) 104 can utilizecommunications protocols, including packet-based and/or datagram-basedprotocols such as internet protocol (IP), transmission control protocol(TCP), user datagram protocol (UDP), or other types of protocols.Moreover, network(s) 104 can also include a number of devices thatfacilitate network communications and/or form a hardware basis for thenetworks, such as switches, routers, gateways, access points, firewalls,base stations, repeaters, backbone devices, and the like.

In some examples, network(s) 104 can further include devices that enableconnection to a wireless network, such as a wireless access point (WAP).Example embodiments support connectivity through WAPs that send andreceive data over various electromagnetic frequencies (e.g., radiofrequencies), including WAPs that support Institute of Electrical andElectronics Engineers (IEEE) 802.11 standards (e.g., 802.11g, 802.11n,and so forth), and other standards.

In various examples, distributed computing resources 102 include devices106(1)-106(N). Embodiments support scenarios where device(s) 106 caninclude one or more computing devices that operate in a cluster or othergrouped embodiment to share resources, balance load, increaseperformance, provide fail-over support or redundancy, or for otherpurposes. Device(s) 106 can belong to a variety of categories or classesof devices such as traditional server-type devices, desktopcomputer-type devices, mobile devices, special purpose-type devices,embedded-type devices, and/or wearable-type devices. Thus, althoughillustrated as desktop computers, device(s) 106 can include a diversevariety of device types and are not limited to a particular type ofdevice. Device(s) 106 can represent, but are not limited to, desktopcomputers, server computers, web-server computers, personal computers,mobile computers, laptop computers, tablet computers, wearablecomputers, implanted computing devices, telecommunication devices,automotive computers, network enabled televisions, thin clients,terminals, personal data assistants (PDAs), game consoles, gamingdevices, work stations, media players, personal video recorders (PVRs),set-top boxes, cameras, integrated components for inclusion in acomputing device, appliances, or any other sort of computing device.

Device(s) 106 can include any type of computing device having one ormore processing unit(s) 108 operably connected to computer-readablemedium (CRM) 110 such as via a bus 112, which in some instances caninclude one or more of a system bus, a data bus, an address bus, a PCIbus, a Mini-PCI bus, and any variety of local, peripheral, and/orindependent buses. Executable instructions stored on CRM 110 caninclude, for example, an operating system 114, a column orderingframework 116, and other modules, programs, or applications that areloadable and executable by processing units(s) 108. Additionally oralternatively, the functionally described herein can be performed, atleast in part, by one or more hardware logic components such asaccelerators. For example, and without limitation, illustrative types ofhardware logic components that can be used include Field-ProgrammableGate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs),Application-Specific Standard Products (ASSPs), System-on-a-Chip systems(SOCs), Complex Programmable Logic Devices (CPLDs), etc. For example, anaccelerator can represent a hybrid device, such as one from ZYLEX orALTERA that includes a CPU course embedded in an FPGA fabric.

Device 106 can also include one or more network interfaces 118 to enablecommunications between computing device 106 and other networked devicessuch as client computing device(s) 120 through which a client can submitquery, and/or user preferences. Such network interface(s) 118 caninclude one or more network interface controllers (NICs) or other typesof transceiver devices to send and receive communications over anetwork. For simplicity, other components are omitted from theillustrated device 106.

Other devices involved in ordering columns of a table for input/outputoptimization, can include client computing devices 120(1)-120(N).Device(s) 120 can belong to a variety of categories or classes ofdevices such as traditional client-type devices, desktop computer-typedevices, mobile devices, special purpose-type devices, embedded-typedevices, and/or wearable-type devices. Thus, although illustrated asmobile computing devices, which can have less computing resources thandevice(s) 106, client computing device(s) 120 can include a diversevariety of device types and are not limited to any particular type ofdevice. Client computing device(s) 120 can include, but are not limitedto, computer navigation type client computing devices 120(1) such assatellite-based navigation systems including global positioning system(GPS) devices and other satellite-based navigation system devices,telecommunication devices such as mobile phone 120(2) mobile phonetablet hybrid 120(3), personal data assistants (PDAs) 120(4), tabletcomputers 120(5), laptop computers such as 120(N), other mobilecomputers, wearable computers, implanted computing devices, desktopcomputers, personal computers, automotive computers, network-enabledtelevisions, thin clients, terminals, game consoles, gaming devices,work stations, media players, personal video recorders (PVRs), set-topboxes, cameras, integrated components for inclusion in a computingdevice, appliances, or any other sort of computing device.

Client computing device(s) 120 can represent any type of computingdevice having one or more processing unit(s) 122 operably connected toCRM 124 such as via a bus 126, which in some instances can include oneor more of a system bus, a data bus, an address bus, a PCI bus, aMini-PCI bus, and any variety of local, peripheral, and/or independentbuses.

Executable instructions stored on CRM 124 can include, for example, anoperating system 128, a user profile system 130 for accessing andapplying user preferences, and other modules, programs, or applicationsthat are loadable and executable by processing units(s) 122.Additionally or alternatively, the functionally described herein can beperformed, at least in part, by one or more hardware logic componentssuch as accelerators. For example, and without limitation, illustrativetypes of hardware logic components that can be used includeField-programmable Gate Arrays (FPGAs), Application-specific IntegratedCircuits (ASICs), Application-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), etc. For example, an accelerator can represent a hybrid device,such as one from ZYLEX or ALTERA that includes a CPU course embedded inan FPGA fabric.

Client computing device 120 can also include one or more networkinterfaces 132 to enable communications between client computing device120 and other networked devices such as other client computing device(s)120 or devices 106 over network(s) 104. Such network interface(s) 132can include one or more network interface controllers (NICs) or othertypes of transceiver devices to send and receive communications over anetwork.

Client computing device 120 can also include a user interface 134 toenable the device 120 to receive user input regarding the columnordering. Examples support the user interface 134 receiving input fromthe user to enable and disable the column ordering system, to specify athreshold for reordering and to apply the column order to subsequentlyadded, and/or pre-existing data. Additionally or alternatively, the userinterface 134 can receive input from a user specifying input/outputquery preferences and/or system parameter preferences, thereby affectingthe resulting column ordering.

In various examples, the user interface 134 can communicate user inputto column ordering framework 116 on device 106, such as, via networkinterface(s) 132 and 118. In such examples, column ordering framework116 may calculate a new column order based at least in part on the userinput.

FIG. 2 is a block diagram depicting an example computing device 200configured to participate in a column ordering for input/outputoptimization system. In some examples, computing device 200 may be onecomputing device of a distributed computing resource, such as a device106 from FIG. 1 . In device(s) 200, processing unit(s) 202, can includeprocessing unit(s) 108 and can represent, for example, a CPU-typeprocessing unit, a GPU-type processing unit, a field-programmable gatearray (FPGA), another class of digital signal processor (DSP), or otherhardware logic components that can, in some instances, be driven by aCPU. For example, and without limitation, illustrative types of hardwarelogic components that can be used include Application-SpecificIntegrated Circuits (ASICs), Application-Specific Standard Products(ASSPs), System-on-a-chip systems (SOCs), Complex Programmable LogicDevices (CPLDs), etc.

In some examples, memory 204 can represent CRM 110 and can storeinstructions executable by the processing unit(s) 202, which asdiscussed above, can represent a processing unit incorporated in device200. Memory 204 can also store instructions executable by externalprocessing units such as by an external CPU, an external GPU, and/orexecutable by an external accelerator, such as an FPGA type accelerator,a DSP type accelerator, or any other internal or external accelerator.In various examples at least one CPU, GPU, and/or accelerator isincorporated in device 200, while in some examples one or more of a CPU,GPU, and/or accelerator is external to device 200.

In the illustrated example, memory 204 also includes a data store 206.In some examples, data store 206 includes data storage such as adatabase, data warehouse, or other type of structured or unstructureddata storage. In some examples, data store 206 includes a corpus and/ora relational database with one or more tables, indices, storedprocedures, and so forth to enable data access such as web tablesincluding one or more of hypertext markup language (HTML) tables,resource description framework (RDF) tables, web ontology language (OWL)tables, and/or extensible markup language (XML) tables, for example.Data store 202 can store data for the operations of processes,applications, components, and/or modules stored in memory 204 and/orexecuted by processing unit(s) and/or accelerator(s) 202. Alternately,some or all of the above-referenced data can be stored on separatememories 208 on board one or more processing unit(s) 202 such as amemory on board a CPU-type processor, a GPU-type processor, an FPGA-typeaccelerator, a DSP-type accelerator, and/or another accelerator.

Device(s) 200 can further include one or more input/output (I/O)interfaces 210 to allow device 200 to communicate with input/outputdevices such as user input devices including peripheral input devices(e.g., a keyboard, a mouse, a pen, a game controller, a voice inputdevice, a touch input device, a gestural input device, and the like)and/or output devices including peripheral output devices (e.g., adisplay, a printer, audio speakers, a haptic output, and the like). Inaddition, in device(s) 200, network interface(s) 212, which can benetwork interface(s) 118, can represent, for example, network interfacecontrollers (NICs) or other types of transceiver devices to send andreceive communications over a network.

In the illustrated example, memory 204 also includes an operating system214, which can be operating system 114. Memory 204 also includes acolumn ordering framework 216, which can be column ordering framework116. Column ordering framework 216 can include one or more modulesand/or APIs, which are illustrated as blocks 218, 220, 222, 224, and226, although this is just an example, and the number can vary higher orlower.

Functionality described associated with blocks 218, 220, 222, 224, and226 can be combined to be performed by a fewer number of modules and/orAPIs or it can be split and performed by a larger number of modulesand/or APIs. For example, block 218 can represent a workload managermodule extraction module with logic to program processing unit 202 ofdevice 200 for extraction of a plurality of query statistics (e.g., anumber of queries, a type of query, a list of columns accessed perquery, and any other statistics relevant to the column orderingoperation). In some examples, the plurality of statistics may be storedon computing device 200, such as, for example, in data store 206. Insome examples, the plurality of statistics can be extracted or receivedfrom a remote computing device, such as via the network interface 212,and stored in data store 206.

The workload manager module may obtain the plurality of statisticsperiodically, such as, for example, at a specified time each day. Insome examples, the periodic interval can be determined by a programmanager (e.g., monthly, weekly, daily). In some examples, the workloadmanager module can obtain or access data when manually directed by theprogram manager.

In some examples, the workload manager module can input the statisticsregarding the processed queries, and can determine a weight of eachquery. For example, a query with a high frequency, e.g., a query thathas been processed 1.5, 2.5, 10 times more than another query, may begiven a higher weight than lower frequency queries. As another example,a query that accesses ten columns can be given a higher weight than aquery that accesses two columns, based at least on the computationalcost of accessing ten columns (i.e., the more computationally expensivea query, the greater improvement to be derived by increasing theefficiency of the query).

Block 220 can represent a user preference module. In various examples,the user preference module may store one or more user profiles. The oneor more user profiles can include user preferences with respect to aquery. For example, a user can input a preference for a particularquery, thereby indicating that the query is important to the user. Theuser preference module can process the user preference information, andassign a high weight to the query, regardless of other query statistics.

In some examples, the user profiles can be associated with a serviceprovider, such as, for example, a cloud service provider. In suchexamples, the user preference module can give preferential treatment tosome users and not to others. For instance, a paying customer of theservice provider can receive preferential treatment over a non-payingcustomer. Additionally or alternatively, the user preference module canreceive the inputs from multiple user profiles, and can assign weightsto queries designated as important to the customers in a consistentmanner. For instance, customers of the same ilk, i.e., all non-payingcustomers, can be treated the same, and queries will be weighted basedon an overall consistency for the customers (e.g., resulting increase inefficiency of searches will be similar for non-paying customers).

Block 222 can represent a hardware simulator module with logic toprogram processing unit 202 for extraction of data related to thehardware component on which the table is stored. The data related to thehardware component can include a type (e.g., electronic, magnetic,optical, mechanical, etc.), performance characteristics (e.g., accesstime, average latency, data rate, etc.), a capacity (e.g., storagesize). For example, the hardware simulator module can determine that thedata is stored on a 2 TB HDD with a 64 MB cache, and a rotation speed of7,200 RPM and a Serial ATA of 6.0 Gb/s, or some other configuration.

In some examples, the hardware simulator module can comprise logic todetermine a seek cost function related to the hardware component. Theseek cost function can abstract the actual properties of the physicalhardware component and can vary across different models, versions and/orstock keeping units (SKUs). In various examples, the hardware simulatormodule can include logic to determine a seek cost for a query byapplying a seek distance (i.e., a distance between columns) to the seekcost function. In such examples, the hardware simulator module candetermine a computational cost of a query for a particular hardwarecomponent.

Block 224 can represent system setting module with logic to programprocessing unit 202 of device 200 for extracting one or more systemparameters associated with the data storage. The system parameters cancomprise a column compression, a row size, a data type, a row type, andany other parameters related to the data storage.

Block 226 can represent an order generating module with logic to programprocessing unit 202 of device 200 for generating a new column orderingstrategy. The order generator module can receive input from one or moremodules including the workload manager module, the user preferencemodule, the hardware simulator module, and/or the system setting module.The input can comprise weighted queries, user preferences, seek costs,and/or system parameters. The order generator module can process theinput, and determine a new column order. In some examples, the ordergenerator module can test the efficacy of the new column order prior toimplementation.

In various examples, block 226 can comprise logic to program processingunit 202 to apply the new column ordering strategy to the table. Theimplementation can apply to subsequently added data and/or to existingdata stored in the table.

Bus 228, which can be bus 112, and which in some instances can includeone or more of a system bus, a data bus, an address bus, a PCI bus, aMini-PCI bus, and any variety of local, peripheral, and/or independentbuses, can operably connect memory 204 to processing unit(s) 202.

FIG. 3 is a dataflow diagram 300 illustrating example interactionsbetween modules of the column ordering framework 302, such as columnordering framework 216. Column ordering framework 302 can process aplurality of data, and output a new column order for optimizinginput/output, such as for a search.

In various examples, workload manager module 304, such as the workloadmanager module depicted in block 218, can receive a plurality of querystatistics associated with one or more queries from data store 306. Thequery statistics can include, but are not limited to, a number ofqueries, a frequency of a query, a type of query, and a list of columnsaccessed per query. In some examples, a big data analytic system (BDAS)308 (e.g., Spark, Scope, MapReduce, etc.) can track all queries to aparticular table, and can continually process and store the statisticsassociated with each query. The BDAS 308 can subsequently send the querystatistics to the workload manager module 304 via the data store 306. Insome examples, the workload manager module 304 can receive the querystatistics directly from the BDAS 308, and can subsequently store themost recent statistics in the data store 306.

The workload manager module 304 can receive the plurality of statisticsassociated with queries periodically, such as, for example, at adesignated time each day, week or month. In some examples, the periodicinterval can be determined by a program manager, which can be internalor external to the workload manager module 304. In some examples, theprogram manager can automatically instruct the BDAS 308 to send updatedquery statistics at a periodic interval. In some examples, the programmanager can manually instruct the BDAS 308 to send the updated querystatistics. For example, the program manager can determine that a higherthan normal number of queries have been asked of a particular table. Theprogram manager can subsequently instruct the BDAS 308 to send anupdated query in between periodic intervals based on the increase inqueries.

In some examples, the workload manager module 304 can input the querystatistics, and can determine a weight of each query, the weight beingan indication of relative importance to the column ordering forinput/output optimization. For example, a query with a high frequency,e.g., a query that has been processed 1.5, 2.5, or 10 or more times morethan another query, can be given a higher weight than lower frequencyqueries. As another example, a query that accesses twenty columns can begiven a higher weight than a query that accesses fewer columns, based atleast on the computational cost of accessing twenty columns (i.e., themore computationally expensive a query, the greater the improvement tobe derived by increasing the efficiency of the query).

The workload manager module 304 can send the weights associated witheach query and relevant statistics, to the order generating module 310.For example, relevant statistics can comprise the columns associatedwith each query, and any other data relevant to generating a new columnorder for input/output optimization.

Order generating module 310 can receive input from one or more modules,such as workload manager module 304, user preference module 312,hardware simulator module 314, and/or system setting module 318, invarious examples. Order generating module 310 may receive input from anycombination of the foregoing modules. For example, order generatingmodule 310 can receive input from workload manager module 304 andhardware simulator module 314. As another example, order generatingmodule 310 can receive input from workload manager module 304, userpreference module 312, hardware simulator module 314, and system settingmodule 318.

In some examples, the order generating module 310 can receive queryweights from a user preference module 312. The user preference module312 can access and/or store user preferences, such as, for example, in auser profile. In various examples, the user preferences can be stored indata store 306, and accessed by the user preference module 312. The userpreferences can comprise an indication that one or more queries isimportant to a particular user. For example, a user can input apreference for a particular query, thereby indicating that the query isimportant to the user.

The user preference module 312 can process the user preferenceinformation, and assign a weight to the query. In some examples, theuser preference module can evaluate received preference information anddetermine a weight based at least in part on the preference information.For example, if the user indicates that a query is of high importance,the user preference module can assign a heavy weight to the query. Foranother example, if the user indicates that the query is of medium tolow importance, the user preference module can assign a lower weight tothe query.

In some examples, the user preference module 312 can evaluate the sourceof the received preference information, and adjust the weight based on aparticular source (i.e., a particular user). For example, a user payingfor a cloud providing service can receive preferential treatment over anon-paying customer. In such an example, the user preference module canincrease the weight of a query preference for the paying customer and/orcan decrease the weight of a query preference for the non-payingcustomer. For another example, users within each paying or non-payingcategories can receive preferential treatment, such as, for example,users within different sub-categories. For instance, a user paying for aplatinum service can receive preferential treatment over a user payingfor a gold or silver service.

Additionally or alternatively, the user preference module 312 canreceive user preference information from a plurality of users, and cancalculate weights based on a consistency scale. The consistency scalecan comprise distribution of improved query return performance acrossall users, a category and/or a sub-category of users. In variousexamples, the user preference module 312 can calculate and/or adjust theweights for queries in such a way that would result in a new columnorder that provides maximum query return performance across all users.In such an example, the user preference module 312 can balance querypreferences across all users, and determine the maximum benefit for thegreatest number of users. In some examples, the user preference module312 can calculate and/or adjust the weights for queries in such a waythat would result in a new column order that provides greater queryreturn performance for some users and not others. For example, two usersof the same sub-category can experience relatively equivalent queryreturn performance; however users of higher sub-categories canexperience better query return performance.

In various examples, the user preference module 312 can send theprocessed weights to order generating module 310. Based at least in parton the importance of queries to users, the order generating module 310can determine a new column order for input/output optimization.

Order generating module 310 can receive input from hardware simulatormodule 314, which can be hardware simulator module represented at block222, can process data related to one or more hardware components onwhich the table is stored. The data related to the hardware componentscan comprise a type (e.g., electronic, magnetic, optical, mechanical,etc.), performance characteristics (e.g., access time, average latency,data rate, etc.), a capacity (e.g., storage size). For example, thehardware simulator module can determine that the data is stored on a 2TB HDD with a 64 MB cache, and a rotation speed of 7,200 RPM and aSerial ATA of 6.0 Gb/s, or some other configuration.

In various examples, the hardware simulator module 314 can receive thedata related to the one or more hardware components from a distributedfile system 316. The distributed file system 316 can comprise a HadoopDistributed File System (HDFS), a Cosmos Distributed File System (CDFS),or any other type of distributed file system capable of representingdata as a two-dimensional table. In some examples, the distributed filesystem 316 can be saved internally to a computing device, such ascomputing device 200. In such an example, the distributed file system316 can be saved in the data store, such as data store 206. In someexamples, the distributed file system 316 can be saved across multiplecomputing devices, such as in distributed computing resource 102. Thedistributed file system 316 can comprise one or more hardwarecomponents, upon which a table is stored, and can provide data relatedto the one or more hardware components to the hardware simulator module314.

The hardware simulator module 314 can process the data, and candetermine a seek cost of each hardware component (e.g., cost of a diskseek). The seek cost of the hardware component can be largely dependenton a seek distance between two columns accessed saved to the hardwarecomponent, such as, for example, columns i and j. The seek distancebetween columns i and j can be calculated as follows:

dist (i,j)=|b(j)−e(i)|

b(i) is the offset of the first byte of the data object i in the file;and e(i) is the offset of the byte next to the last byte of i. Thus,e(i)=b(i)+s(i).

Additionally, the hardware simulator module 314 can determine a seekcost function related to the hardware component. The seek cost functionabstracts the actual properties of the physical hardware component andcan vary across different stock keeping units (SKUs). In variousexamples, such as examples storing tables on HDDs, the hardwaresimulator module 314 can apply curve fitting to determine the seek costfunction related to the hardware components. In such examples, thehardware simulator module 314 can first select a set of blocks randomlydistributed on different zones (i.e., different offsets) of the disks ina homogeneous cluster, the homogeneous cluster being disks of the samehardware. The hardware simulator module 314 can then performread-and-seek operations of different seek distances in each blocks ofthe set of blocks, and can record the elapsed time (i.e., seek time) ofeach seek operation. The hardware simulator module 314 can calculate anaverage seek time. After calculating an average seek time, the hardwaresimulator module 314 can fit the curve with the seek distances and thecorresponding average seek time.

While the above described example is specific to HDDs, it is understoodthat other seek cost function processes can be implemented by evaluatingother hardware components, such as, for example, flash drives. For eachtype of hardware component, the seek function can be different. Thus,for heterogeneous hardware components in a storage system, the hardwaresimulator can determine a seek function for each type of hardwarecomponent. However, to reduce computational complexity, in someexamples, the hardware simulator module 314 can apply a generic seekfunction to heterogeneous hardware components. After determining theseek function, the hardware simulator module 314 can calculate the seekcost from one column to another.

To calculate the seek cost between two columns, the hardware simulatormodule 314 can apply the seek distance, as described above, to the costfunction. Thus, given the two columns i and j, the seek cost from i to jcan be modeled as:

Cost(i,j)=f(dist(i, j)).

The hardware simulator module 314 can send the seek cost of the hardwarecomponent to the order generating module 310. In some examples, thehardware simulator module 314 can additionally send the seek costfunction of the hardware component to the order generating module 310.However, the algorithms of the order generating module 310 are generallyagnostic to the seek function, and thus apply generally to differentstorage media.

In various examples, the order generating module 310 can also receivesystem parameters from system setting module 318. In various examples,the system setting module 318 can receive system parameters fromdistributed file system 316. The system parameters can comprise a columncompression, a row size, a data type, a row type, and any otherparameters related to the data storage.

As outlined above, order generating module 310 receives input fromworkload manager module 304, user preference module 312, hardwaresimulator module 314, and system setting module 316. The input cancomprise weighted queries, user preferences, seek costs, and/or systemparameters. The order generator module can process the input, anddetermine a new column order.

Column Ordering Problem

Very large tables, with hundreds or thousands of columns, can requiregreat computational costs to return a result from a query. The followingdiscussion explains the column ordering problem, as a function of thequery seek cost.

A query q generally accesses a subset of columns, which can berepresented as:

C_(q)={c_(q,1), c_(q,2), . . . c_(q,m)},

where C_(q) represents the columns that q must access following a globalorder of S, where m≤n, and c_(q,i) represents the i^(th) column accessedby q. Therefore, c_(q,i) must appear before c_(q,j) in S if i<j. Inother words, the accessed columns for q follow the same order of thecolumn placement strategy S. The read task for processing the query qcan then access the columns specified by q one by one. The query seekcost may thus be expressed as follows:

Cost(q,S)=Σ_(i=0) ^(m−1)Cost(c _(q,i) ,c _(q,i+1)),

S={c₁, c₂, . . . , c_(n)} being the column placement strategy.

Because the table is partitioned into a number of row groups, and eachrow group follows the same column placement strategy, S, the seek costof the query over the whole table will be proportional to the seek costof a row group. Thus, the query seek cost above can be applied over thewhole table. When the number of columns is small (e.g., <50), the queryseek cost may not be significant. However, with larger tables, such as,for example, tables with greater than 1000 columns, the query seek costbecomes a significant part of the I/O cost. At least in part to decreasethe I/O cost, order generation module 310 determines a new column orderto optimize input/output efficiency.

To find an optimal column placement strategy S*={c₁, c₂, . . . , c_(n)},such that the overall query seek cost of Q is minimized:

${S^{*} = {\arg\begin{matrix}\min \\S\end{matrix}{\sum_{q \in Q}\left( {w_{q} \times {Cost}\left( {q,S} \right)} \right)}}},$

where q∈Q represents a weighted vector for the weight w_(q) of eachquery. As described above, the weight w_(q) for each query can representa frequency and/or importance, and can be based on input from theworkload manager module 304 and/or the user preference module 312.Absent specification or input regarding the weight of a particularquery, the particular query can be assigned a default w_(q)=1.

Column Order Optimization

The order generating module 310 can process inputs through one or morealgorithms (e.g., Simulated Annealing Based Column Ordering Algorithm(SCOA), Genetic Algorithm (GA), Frequency Algorithm, and the like) anddetermine a new column order with increased input/output efficiency(i.e., lower computational cost). As discussed above, the inputs can befrom one or more of the workload management module 304, user preferencemodule 312, hardware simulator module 314, and system setting module318.

In some examples, the order generating module 310 can apply the SCOAalgorithm to solve the order generating problem described above.Simulated annealing (SA), the basis of the SCOA algorithm, includes aniterative procedure that continuously updates the current state untilthe system reaches a state that is sufficient (e.g., a predeterminedthreshold), or until a given computation budget has been exhausted. Ineach iteration, SA considers some neighboring state S′ of the currentstate S. A neighbor candidate state is generated by perturbing thecurrent state. An Accept function can determine whether the neighborstate should be accepted or not. The Accept function can utilize aTemperature function and an Energy function for making this decision.The temperature returned by the Temperature function decreases (i.e.,cools down) during the iteration process following a specificdistribution, which is called an annealing schedule. The Energy functioncan be used to calculate the energy of a state. The objective of SA isto achieve a state with lowest energy. Thus, the neighbor state with thelowest energy can generally be more likely to be accepted.

With regard to the column ordering problem, a specific column order canconstitute the state of the algorithm. For example, if there is a ‘good’column order S, it is likely that there is a better state S′ ‘close’ toS (i.e., S′ and S are related to one another). Additionally the queryseek cost of a column order in SCOA servers as the Energy function ofSA.

An AcceptColumnOrder function of SCOA can make a decision betweenaccepting or rejecting a neighbor state probabilistically. Thedistribution of the acceptance probability is determined by theannealing schedule. This can ensure that SCOA is not likely to trap intoan undesirable local minimum at the beginning. For example, in at leastone implementation, the temperature is relatively high at the beginning.However, as time goes by, the temperature drops exponentially, and theseek cost of the workload decreases accordingly. Thus, when thetemperature is high, SCOA probabilistically explores the search space toavoid getting stuck in a local minimum. The probabilistic explorationcan result in some fluctuations at the beginning, but as the temperaturecools, the SCOA converges to generate a near optimal column order.

The SCOA algorithm can be represented as shown in TABLE 1.

TABLE 1 Requires: The set of queries, Q = {q₁, q₂, ..., q_(m)} Theinitial column order, S₀ = {c₁, c₂, ..., c_(n)} Column and column sizepair list, M = {(c₁, s(c₁)), (c₂, s(c₂)), ..., (c_(n), s(c_(n)))}, inwhich s(c_(i)) is the size of column c_(i). Ensures: The optimizedcolumn order S; S := S₀, e := SeekCost(S₀), t := init_temperature; for k:= 1 to k_(max) do   t := Temperature(t, cooling _rate );   S′ :=Neighbor(S);   e′ := SeekCost(Q, M, S′);   if AcceptColumnOrder(e, e′,t) then   S := S′;   e := e′;  end if end for return S;

Given a workload Q, an initial column order S₀ and the data size of eachcolumn, SCOA returns a near optimal column ordering strategy S, so thatthe total I/O cost of Q is minimized. In SCOA, the main loop (paragraphs[0094]-[0102]) shows the iterative search process which is based on acooling schedule. The Temperature function is the controlling functionof the annealing schedule. In SCOA, the temperature can shrink at a rateof (1-cooling rate). The Neighbor(S) function is to obtain a candidatestate that is transformed from the current column order S. A neighborcolumn order can be generated by swapping the location of two randomlychosen columns in the current column order. The seek cost of a columnorder can computed by the Seek Cost function input from the workloadmanager module 304.

The parameter represents the maximum number of iterations in the mainloop, which controls the termination of SCOA. In various examples,k_(max) is set large enough to ensure thatinitial_temperature*(1-cooling_rate)^(kmax) nears zero. As thetemperature approaches zero, it is unlikely to accept a worse candidatestate with a higher seek cost, thus leading to convergence.

The AcceptColumnOrder function can be represented as shown in TABLE 2.

TABLE 2 Require: The seek cost of S, e; The seek cost of S′, e′; Thetemperature, t; Ensure: If accept S′return true, else return false; ife′ < e then  return true else if random(0,1) < Prob(e, e′, t) then return true; else  Return false; end if

As described in the algorithms of TABLE 1 and TABLE 2, a neighborcandidate state whose seek cost is lower than that of the current statewill be directly accepted. Otherwise, it will be conditionally acceptedwith a threshold returned by the Prob function which is set as

${{{Prob}\left( {e,e^{\prime},t} \right)} = {\exp\left( \frac{e - e^{\prime}}{t} \right)}},$

and random(0,1) generates a random number between 0 and 1 under auniform distribution.

The SCOA is but one illustrative example algorithm that the ordergenerating module 310 can apply to the one or more inputs. Other exampleexamples can use a genetic algorithm, a frequency algorithm, or anyother algorithm for column ordering to achieve input/outputoptimization.

In various examples, after determining a new column order, the ordergenerator module 310 can test the efficacy of the new column order priorto implementation. For example, the order generating module 310 can takea sample set of queries, and test for a decreased overall query seekcost.

The order generator module 310 can send the new column order to thedistributed file system 316 for implementation. The order generatormodule 310 can send the new column order at a pre-determined time (e.g.,daily, weekly, monthly), when the cost savings of the new orderincreases above a threshold level, and/or upon input from a userdirecting the update. In some examples, the order generating module 310can additionally send instructions to implement the new column order tosubsequently added data and/or pre-existing data. An instruction toimplement the new column order to pre-existing data requires thedistributed file system to re-save the table, in its entirety, therebymaking the instruction computationally expensive. As such, the ordergenerating module 310 may, in some examples, maintain a re-save costthreshold value, above which it would be more efficient to reordercolumns, i.e., despite the computational cost of re-saving the totalityof the table, the system would enjoy an overall increase in input/outputefficiency and decrease in query seek cost.

Illustrative Processes

FIGS. 4, 5 and 6 are flow diagrams depicting example processes for acolumn ordering for input/output optimization. The operations of theexample process are illustrated in individual blocks and summarized withreference to those blocks. The processes are illustrated as logicalflows of blocks, each block of which can represent one or moreoperations that can be implemented in hardware, software, or acombination thereof.

FIG. 4 is a flow diagram that illustrates a process flow 400 ofdetermining a new column order using a column ordering for input/outputoptimization system.

At block 402, a computing device, such as computing device 200, candetermine a first order for columns of a table.

At block 404, the computing device can receive one or more querystatistics associated with the table. The one or more query statisticscan comprise a number of queries, a type of query, a list of columnsaccessed per query, and any other statistics relevant to the columnordering operation. In some examples, the computing device can receivethe one or more query statistics periodically, such as, for example,daily. In some examples, the computing device can receive the one ormore query statistics in response to a manual request for updatedstatistics from the computing system or from a user via the computingsystem.

At block 406, the computing device can process the query statisticsreceived at block 404, and can determine a first seek cost of the one ormore queries. For example, a first query may access three columns, andmay have a first computational cost. As another example, a second querymay access ten columns, and may have a second computational cost, thesecond computational cost being higher than the first computationalcost. As yet another example, a third query may access two columns, andmay have a third computational cost, the third computational cost beinghigher than the first and the second computational costs.

At block 408, the computing device can receive one or more inputs. Theone or more inputs may include user preference data (e.g., a preferenceof one or more queries, and/or system parameters. In some examples, theuser preference data can include instructions from the user to thesystem. For instance, the user preference data can include aninstruction to enable or disable the column ordering system, tospecifying a threshold for reordering, and/or to apply the column orderto subsequently added, and/or pre-existing data.

In some examples, the one or more inputs can include hardware dataassociated with one or more hardware components storing the table. Thedata related to the one or more hardware component can include a type,performance characteristics, a capacity and/or other data related to thehardware component. For example, the hardware simulator module candetermine that the data is stored on a 2 TB HDD with a 64 MB cache, anda rotation speed of 7,200 RPM and a Serial ATA of 6.0 Gb/s, or someother configuration.

In some examples, the one or more inputs can include system parametersassociated with the distributed file system. The system parameters cancomprise a column compression, a row size, a data type, a row type, andany other parameters related to the data storage within the distributedfile system.

At block 410, the computing device can determine a second order forcolumns of the table based on one or more of the query statistics, theuser preference data, the hardware data, the seek cost, and the systemparameters. In some examples, the computing device can determine thesecond column order using a simulated annealing column orderingalgorithm (SCOA). In some examples, the computing device can determinethe second column order using a genetic algorithm or a frequencyalgorithm.

At block 412, the computing device can apply the second order forcolumns of the table to create an ordered table. In some examples, theapplication can include sending the second order to a distributed filesystem with an instruction to apply the second order. The instruction toapply can include an instruction to apply the second column order to thecolumns of the table and/or to data that is added to the tablesubsequent to the determination of a second order for the columns.

At block 414, the computing device may receive an input/output request,such as a search query.

At block 416, the computing device may search the ordered table for aresult to the search query.

FIG. 5 is a flow diagram that illustrates a process flow 500 ofdetermining a seek cost in the column ordering for input/outputoptimization process.

At block 502, similar to block 404, the computing device can receive oneor more query statistics associated with the table. The one or morequery statistics can comprise a number of queries, a type of query, alist of columns accessed per query, and any other statistics relevant tothe column ordering operation. In some examples, the computing devicecan receive the one or more query statistics periodically, such as, forexample, daily. In some examples, the computing device can receive theone or more query statistics in response to a manual request for updatedstatistics from the computing system or from a user via the computingsystem.

At block 504, the computing device can determine a seek function for oneor more hardware components storing the table. The seek function canabstract the actual properties of the physical hardware component andcan vary across different models, versions and/or stock keeping units(SKUs).

At block 506, the computing device can determine a distance betweenrelated columns. In some examples, the related columns can be two ormore columns that are likely to be accessed together in a query. Therelated columns can be determined by the one or more query statisticsreceived at block 404.

At block 508, the computing device can apply the distance betweenrelated columns to the seek function.

At block 510, as a result of applying the distance to the seek function,the computing device can determine a seek cost (i.e., a computationalcost) for a query, the seek cost being hardware-specific.

FIG. 6 is a flow diagram that illustrates a process flow 600 ofdetermining a new column order using a column ordering for input/outputoptimization system.

At block 602, the computing device can access a table with a pluralityof columns.

At block 604, the computing device can identify at least two relatedcolumns. The related columns can be any two or more columns likely to beaccessed together in a query. In some examples, the at least two relatedcolumns may be identified by one or more query statistics.

At block 606, the computing device may determine whether the relatedcolumns are close together. In various examples, a determination ofwhether columns are close together can be based, at least in part, onwhether the related columns are physically located close together. Insome examples, a determination of whether the related columns are closetogether can be based, at least in part, on a determination that therelated columns can be accessed quickly together. The determination ofwhether the related columns can be accessed quickly together can bebased on a seek cost to access the related columns. The seek cost can bebased on a seek function of the hardware components storing the table,and a distance between the columns.

At block 608, the computing device can determine that the relatedcolumns are located close to one another, and may maintain the positionof the related columns.

At block 610, the computing device can determine that the relatedcolumns are not located close to one another, and may determine a neworder for the plurality of columns. In some examples, the new order maybe determined based upon one or more of query statistics, userpreferences, hardware data, and/or system parameters.

At block 612, the computing device can apply the new order to create anordered table, the ordered table configured with columns ordered tooptimize input/output efficiency.

In some examples, the application of the new order may include sendingimplementation instructions to a distributed file system storing thetable. The implementation instructions can include an instruction toapply the new order to the columns of the table and/or to data that isadded to the table subsequent to the determination of the new order.

In some examples, the implementation instructions can be based at leastin part on user preference data. For example, a user can enable ordisable the column ordering process. In such an example, upon the userenabling the column ordering process, the computing device can calculatea new order and send a signal to the distributed file system toimplement. For another example, a user can determine a threshold, aboveor below which computing device automatically re-calculates an order andsends it to the distributed file system to implement.

At block 614, the computing device may receive an input/output request,such as a search query.

At block 616, the computing device may search the ordered table for aresult to the search query.

EXAMPLE CLAUSES

A: A computer-implemented method comprising: ascertaining a first orderof columns of a table, the table comprising a plurality of columns;receiving statistics associated with one or more queries; determining afirst cost for the one or more queries based at least in part on thestatistics; determining a second order of columns of the table, thesecond order of columns having a lower cost than the first cost; andapplying the second order of columns to create an ordered table.

B: A method as paragraph A describes, wherein the determining the firstcost for the one or more queries comprises: determining a seek functionfor a hardware storing the table, determining a distance between atleast two related columns of the plurality of columns; applying thedistance to the seek function; and determining a cost.

C: A method as either paragraph A or paragraph B describes, furthercomprising: receiving one or more inputs related to at least one of: auser preference associated with the table; a hardware component; and asystem parameter, wherein the determining the second order of columns ofthe table is based at least in part on the one or more inputs.

D: A method as paragraphs A-C describe, wherein the ordered tablecomprises: the table configured in the first column order; and dataadded subsequent to a determination of the second order of columns.

E: A method as paragraphs A-C describe, wherein the ordered tablecomprises: the table and data added subsequent to a determination of thesecond order of columns.

F: A method as paragraphs A-C describe, further comprising: receiving aquery; and searching the ordered table for a result to the query.

G: A computer-readable medium having thereon computer-executableinstructions that responsive to execution configure a computer toperform a method as any one of paragraphs A-F describes.

H: A device or system comprising: a processor; and a computer-readablemedium coupled to the processor, the computer-readable medium includinginstructions to configure one or more devices to perform a method as anyof paragraphs A-F describes.

I: A device or system comprising: means for processing; and means forstoring coupled to the means for processing, the means for storing,storing instructions to configure one or more devices to perform amethod as any of paragraphs A-F describes.

J: A computer-implemented method comprising: accessing a tablecomprising a first plurality of columns; identifying at least tworelated columns of the first plurality of columns; determining whetherthe at least two related columns of the first plurality of columns areclose to one another; responsive to determining the at least two relatedcolumns of the first plurality of columns are not close to one another,determining a new order for the first plurality of columns based atleast in part on locating the at least two related columns close to oneanother; and applying the new order to a second plurality of columns tocreate an ordered table, the second plurality of columns being added tothe table subsequent to the a determination of the new order.

K: A method as paragraph J describes, wherein the identification of theat least two related columns comprises a determination that the at leasttwo related columns will likely be accessed together.

L: A method as paragraphs J-K describe, wherein the determining whetherthe at least two related columns of the plurality of columns are closeto one another is based at least in part on a distance model.

M: A method as paragraphs J-L describe, further comprising: identifyinga column in the table not relevant to the search query; and removing thecolumn from the table prior to the searching the table for the result tothe search query.

N: A method as paragraphs J-M describe, further comprising: applying thenew order to the first plurality of columns.

O: A method as paragraphs J-N describe, wherein the identifying the atleast two related columns of the plurality of columns comprises:receiving statistics associated with a query; and identifying two ormore columns accessed in response to the query.

P: A method as paragraphs J-O describe, further comprising: receiving aquery; and searching the ordered table for a result to the query.

Q: A computer-readable medium having thereon computer-executableinstructions that responsive to execution configure a computer toperform a method as any one of paragraphs J-P describes.

R: A device or system comprising: a processor; and a computer-readablemedium coupled to the processor, the computer-readable medium includinginstructions to configure one or more devices to perform a method as anyof paragraphs J-P describes.

S: A device or system comprising: means for processing; and means forstoring coupled to the means for processing, the means for storing,storing instructions to configure one or more devices to perform amethod as any of paragraphs J-P describes.

T: A computer-readable medium having thereon computer-executableinstructions, the computer-executable instructions responsive toexecution configuring a device to perform operations comprising:accessing a table comprising a first plurality of columns; identifyingat least two related columns of the first plurality of columns;determining whether the at least two related columns of the firstplurality of columns are close to one another; responsive to determiningthe at least two related columns of the first plurality of columns arenot close to one another, determining a new order for the firstplurality of columns based at least in part on locating the at least tworelated columns close to one another; and applying the new order to asecond plurality of columns to create an ordered table, the secondplurality of columns being added to the table subsequent to thedetermining the new order for the first plurality of columns.

U: A computer-readable medium as paragraph T describes, wherein theidentification of the at least two related columns comprises adetermination that the at least two related columns will likely beaccessed together while accessing the ordered table for a result to aquery.

V: A computer-readable medium as paragraphs T-U describe, wherein thedetermining whether the at least two related columns of the plurality ofcolumns are close to one another is based at least in part on a seekcost.

W: A computer-readable medium as paragraphs T-V describe, furthercomprising: identifying a column in the table not relevant to a query;and removing the column from the table prior to searching the table forthe result to the query.

X: A computer-readable medium as paragraphs T-W describe, furthercomprising:

applying the new order to the first plurality of columns.

Y: A computer-readable medium as paragraphs T-X describe, wherein theidentifying the at least two related columns of the plurality of columnscomprises: receiving statistics associated with a query; and identifyingtwo or more columns accessed in response to the query.

Z: A computer-readable medium as paragraphs T-Y describe, the operationsfurther comprising: receiving a query; and searching the ordered tablefor a result to the query.

AA: A device or system comprising: a processor; and a computer-readablemedium as any of paragraphs T-Z describes coupled to the processor.

AB: A device or system comprising: means for processing; and means forstoring coupled to the means for processing, the means for storing,storing instructions to configure one or more devices to perform amethod as any of paragraphs T-Z describes.

AC: A system comprising: a processor; a memory storing a column orderingframework for actuation by the processor, the column ordering frameworkcomprising: a workload manager module configured and process dataregarding a first column layout of tabular storage to determine a queryworkload associated with the first column layout; and an ordergenerating module configured to calculate a second column layout for thetabular storage based at least in part on the query workload.

AD: A system as paragraph AC describes, wherein the data comprises atleast one of: a frequency of a query; a list of columns accessed for thequery; a time stamp for the query; or a seek cost for the query.

AE: A system as paragraphs AC-AD describe, wherein the order generatingmodule is further configured to apply the second column layout of thetabular storage to data added subsequent to the calculation of thesecond column layout.

AF: A system as paragraphs AC-AE describe, wherein the column orderingframework further comprises: a user preference module configured toreceive and process user preference inputs regarding the table; ahardware simulator module configured to receive data associated with thehardware storing the table; and a system setting module configured toreceive system settings related to the table.

AG: A system as paragraph AF describes, wherein the user preferenceinputs comprise at least one of: an input to enable the column orderingframework; an input to disable the column ordering framework; apreference for one or more queries; or an efficiency threshold forcalculating the second column layout of the table.

AH: A computer-readable medium having thereon computer-executableinstructions that responsive to execution configure a computer toperform a system as any one of paragraphs AC-AG describes.

AI: A device or method comprising: a processor; and a computer-readablemedium coupled to the processor, the computer-readable medium includinginstructions to configure one or more devices to perform a method as anyof paragraphs AC-AG describes.

AJ: A device or method comprising: means for processing; and means forstoring coupled to the means for processing, the means for storing,storing instructions to configure one or more devices to perform amethod as any of paragraphs AC-AG describes.

AK: A device comprising: a workload manager module configured to receiveand process the data regarding a first order of columns of a table; auser preference module configured to receive and process user preferenceinputs regarding the table; a hardware simulator module configured toreceive data associated with the hardware storing the table; a systemsetting module configured to receive system settings related to thetable; and an order generating module configured to receive inputs fromone or more of the workload manager module, the user preference module,the hardware simulator module, and the system setting module, andcalculate a second order of columns of the table.

AL: A device as paragraph AK describes, wherein the data comprises atleast one of: a frequency of a query; a list of the columns accessed forthe query; a time stamp for the query; or a seek cost for the query.

AM: A device as paragraphs AK-AL describe, wherein the order generatingmodule is further configured to apply the second order of columns of thetable to subsequently added data.

AN: A device as paragraphs AK-AM describe, wherein the user preferenceinputs comprise at least one of: an input to enable the column orderingframework; an input to disable the column ordering framework; apreference for one or more queries; or an efficiency threshold forcalculating the second order of columns of the table.

CONCLUSION

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as illustrative forms ofimplementing the claims.

Conditional language such as, among others, “can,” “could,” “might” or“may,” unless specifically stated otherwise, are otherwise understoodwithin the context as used in general to present that certain examplesinclude, while other examples do not include, certain features, elementsand/or steps. Thus, such conditional language is not generally intendedto imply that features, elements and/or steps are in any way requiredfor one or more examples or that one or more examples necessarilyinclude logic for deciding, with or without user input or prompting,whether these features, elements and/or steps are included or are to beperformed in any particular example.

Conjunctive language such as the phrase “at least one of X, Y or Z,”unless specifically stated otherwise, is to be understood to presentthat an item, term, etc. can be either X, Y, or Z, or a combinationthereof.

Any routine descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode that include one or more executable instructions for implementingspecific logical functions or elements in the routine. Alternateexamples are included within the scope of the examples described hereinin which elements or functions can be deleted, or executed out of orderfrom that shown or discussed, including substantially synchronously orin reverse order, depending on the functionality involved as would beunderstood by those skilled in the art.

It should be emphasized that many variations and modifications can bemade to the above-described examples, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

1.-20. (canceled)
 21. A system comprising: a processor; memory storinginstructions that, when executed by the processor, causes the system toperform operations comprising: identifying two related columns of aplurality of columns stored in a data storage table, wherein theplurality of columns is arranged in a first order; determining a firstseek cost to access the two related columns; determining the two relatedcolumns are not located close to each other based on the first seekcost; in response to determining the two related columns are not locatedclose to each other, determining a second order for the plurality ofcolumns based on one or more storage system parameters associated withthe data storage table; and arranging the plurality of columns accordingto the second order.
 22. The system of claim 21, wherein identifying tworelated columns comprises a determination that the two related columnsare likely to be accessed together in a query.
 23. The system of claim22, wherein the determination that the two related columns are likely tobe accessed together in the query is based on query statisticscomprising at least one of: a number of queries; a type of query; or alist of columns accessed per query.
 24. The system of claim 21, whereinthe first seek cost identifies an amount of time used to access a secondcolumn of the two related columns after accessing a first column of thetwo related columns.
 25. The system of claim 24, wherein determining thetwo related columns are not located close to each other comprisesdetermining that the amount of time used to access the second columnafter accessing the first column exceeds a threshold time value.
 26. Thesystem of claim 24, wherein the amount of time used to access the secondcolumn after accessing the first column is based on at least one of atrack seek time or a disk rotation time.
 27. The system of claim 21,wherein determining a second order for the plurality of columnscomprises determining that a second seek cost for accessing the tworelated columns in the second order is less than the first seek cost foraccessing the two related columns in the first order.
 28. The system ofclaim 21, wherein arranging the plurality of columns according to thesecond order comprises arranging the two related columns such that thetwo related columns are located close together.
 29. The system of claim21, wherein arranging the two related columns such that the two relatedcolumns are located close together comprises storing the two relatedcolumns adjacent to one another in the data storage table.
 30. Thesystem of claim 21, wherein the one or more storage system parameterscomprise at least one of: a column compression; a row size; a data type;or a row type.
 31. The system of claim 21, wherein arranging theplurality of columns according to the second order comprises:determining that at least one of the one or more storage systemparameters exceeds a threshold value; and causing the system toautomatically arrange the plurality of columns based on exceeding thethreshold value.
 32. The system of claim 21, the operations furthercomprising: subsequent to arranging the plurality of columns accordingto the second order, receiving a search query; and performing the searchquery in accordance with the second order.
 33. A method comprising:identifying two related columns of a plurality of columns stored in adata storage table, wherein the plurality of columns is arranged in afirst order; determining a first seek cost to access the two relatedcolumns; determining the two related columns are not located close toeach other based on the first seek cost; in response to determining thetwo related columns are not located close to each other, determining asecond order for the plurality of columns based on one or more storagesystem parameters associated with the data storage table; and arrangingthe plurality of columns according to the second order.
 34. The methodof claim 33, wherein determining the first seek cost comprises:receiving one or more query statistics for the data storage table; andprocessing the one or more query statistics to determine the first seekcost.
 35. The method of claim 34, wherein processing the one or morequery statistics comprises determining seek costs for a set of queries.36. The method of claim 34, wherein the second order is furtherdetermined based on the one or more query statistics.
 37. The method ofclaim 33, the method further comprising: receiving user preference dataincluding instructions to enable or disable automatic column ordering ofthe plurality of columns; and arranging the plurality of columnsaccording to the second order based on the user preference data.
 38. Themethod of claim 33, the method further comprising: determining a secondseek cost to access the two related columns in the second order; andarranging the plurality of columns according to the second order whenthe second seek cost is less than the first seek cost.
 39. The method ofclaim 33, wherein the two related columns are closer to each other inthe second order than in the first order.
 40. A device comprising: aprocessor; memory storing instructions that, when executed by theprocessor, causes the device to perform operations comprising:identifying two related columns of a plurality of columns stored in adata storage table, wherein the plurality of columns is arranged in afirst order; determining a seek cost to access the two related columns;determining the two related columns are not located close to each otherbased on the seek cost; in response to determining the two relatedcolumns are not located close to each other, determining a second orderfor the plurality of columns based on one or more storage systemparameters associated with the data storage table; and arranging theplurality of columns according to the second order.